Apparatus and method for abstracting motion picture shape descriptor including statistical characteristic of still picture shape descriptor, and video indexing system and method using the same

ABSTRACT

An apparatus and method for abstracting motion picture shape descriptor including statistical characteristics of still picture shape descriptor, and video indexing system and method using the same. The video indexing system includes: Segmenting means for segmenting video; Abstracting means for abstracting motion picture shape descriptor information from the segmented video information; and Storing means for storing the motion picture shape descriptor information as metadata.

TECHNICAL FIELD

The present invention relates to an apparatus and method for abstractingmotion picture shape descriptors having statistical characteristics ofstill picture shape descriptors, a video indexing apparatus using themotion picture shape descriptor abstracting apparatus and method, and acomputer-readable recording medium for recording a program thatimplements the motion picture shape descriptor abstracting method.

BACKGROUND ART

Increasing amount of video and audio data calls for technologies forretrieving and managing the data efficiently. One of these technologiesis a multimedia indexing technique for abstracting indexing informationrepresenting multimedia data to be used for data retrieval andsearching.

Currently, with respect to a still picture, color histograms, shapedescriptors and/or texture descriptors are used to abstract indexinginformation representing multimedia data, and for audio data, spectrumdescriptors are used. With respect to a motion picture, motioninformation descriptors using motion vectors and/or orbit descriptors ofobjects are used. However, the descriptors of the conventionaltechnologies are not those descriptors used to dynamically index shapeinformation of objects within a video.

In addition, as one dynamic. indexing method for indexing the dynamicchange of shape data, there is a method indexing the shape informationof an object from the entire still pictures that compose a motionpicture or from some representative still pictures by using theconventional still picture shape information indexing method. However,this method has a shortcoming that the data storing and retrievingefficiency is poor, because the amount of indexing information isincreased, as the number of still pictures used for abstracting shapedata is increased.

DISCLOSURE OF INVENTION

It is, therefore, an object of the present invention to provide anapparatus and method for abstracting motion picture shape descriptors byabstracting still picture shape descriptors from the still pictures ofan object that compose a motion picture and abstracting motion pictureshape descriptors having statistical characteristics from the abstractedstill picture shape descriptors to use them as video indexinginformation, a video indexing system using the motion picture shapedescriptor abstracting apparatus and method, and a computer-readablerecording medium for recording a program that implements the motionpicture shape descriptor abstracting method.

In accordance with one aspect of the present invention, there isprovided a system for retrieving motion picture, comprising: a motionpicture segmentation means for segmenting motion picture temporally; amotion picture shape descriptor abstracting means for abstracting amotion picture shape descriptor from the segmented motion picture data;and a motion picture metadata storing means for storing the motionpicture shape descriptor as metadata.

BRIEF DESCRIPTION OF DRAWINGS

The above and other objects and features of the present invention willbecome apparent from the following description of the preferredembodiments given in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram showing a motion picture shape descriptorapparatus and a motion picture retrieving system in accordance with anembodiment of the present invention;

FIG. 2 is a block diagram illustrating the motion picture shapedescriptor abstracter of FIG. 1 in accordance with the embodiment of thepresent invention;

FIG. 3 is a table showing the metadata stored in a motion picturemetadata database for storing motion picture shape descriptors inaccordance with the embodiment of the present invention; and

FIG. 4 is a flow chart describing a method for abstracting motionpicture shape descriptors in accordance with the embodiment of thepresent invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Generally, shape descriptors of objects for a still picture includeoutline-based shape descriptors and region-based shape descriptors. Thepresent invention suggests ‘a motion picture shape descriptor,’ whichrefers to a descriptor obtained by abstracting shape descriptors,including outline-based shape descriptors or region-based shapedescriptors, from the respective still pictures of objects composing amotion picture, and processing the abstracted shape descriptorsstatistically. The statistically processed motion picture shapedescriptors, i.e., statistical characteristic descriptors, have momentcharacteristics, such as mean and variance.

Following is a process for abstracting motion picture shape descriptorsin a statistical shape vector descriptor abstracter.

The shape sequence of an input object is expressed as SS={s₁, s₂, s₃, .. . , s_(n)}. Here, s_(m) denotes an m^(th) shape. A sequence of stillpicture shape descriptor SD={sd₁, sd₂, sd₃, . . . , sd_(n)} is obtainedwith respect to each shape from the above shape sequence by using theconventional still picture shape descriptors, e.g., region-based ones oroutline-based ones. Here, sd_(m) is a still picture shape descriptorabstracted from an m^(th) shape s_(m). The still picture shapedescriptor sd_(m) is generally expressed as an equation in the form of avector, i.e., Equation 1 below.sd _(m) ={sd _(m)(1), sd _(m)(2), sd _(m)(3), . . . , sd _(m)(l)}  Eq. 1wherein l denotes the number of elements that form the vector, andsd_(m)(p) represents a p^(th) element.

The present invention forms a motion picture shape descriptor by usingthe sequence SD of the still picture shape descriptor and abstractingfour shape descriptors (1) to (4), enumerated below.

(1) Mean Shape Descriptor

Mean shape descriptor sd^(av)={sd^(av)(1), sd^(av)(2), sd^(av)(3), . . ., sd^(av)(l)} is abstracted as follows. An m^(th) element sd^(av)(m) isthe mean value of the m^(th) element of each of n number of shapedescriptors that forms SD={sd1, sd2, sd3, . . . , sdn}. It can beobtained based on Equation 2.sd ^(av)(m)=(Σ_(i=l to n) sd _(i)(m))/n  Eq. 2

(2) Variance Shape Descriptor

The variance shape descriptor sd^(var)={sd^(var)(1), sd^(var)(2),sd^(var)(3), . . . , sd^(var)(l)} is abstracted as follows. That is, anm^(th) element sd^(var)(m) is a variance value of the m^(th) element ofeach of n number of shape descriptors that form SD={sd1, sd2, sd3, . . ., sdn}. It can be obtained based on Equation 3.sd ^(var)(m)=(Σ_(i=l to n)(sd _(i)(m)−sd ^(av)(m))²)/n/(n−1)  Eq. 3

(3) Standard Deviation Shape Descriptor

The standard deviation shape descriptor sd^(std)={sd^(std)(1),sd^(std)(2), sd^(std)(3), . . . , sd^(std)(l)} is abstracted as follows.That is, an m^(th) element sd^(std)(m) is a standard deviation value ofthe m^(th) element of each of n number of shape descriptors that formSD={sd1, sd2, sd3, . . . , sdn}. It can be obtained based on Equation 4.sd ^(std)(m)=sqrt(Σ_(i=l to n) (sd _(i)(m)−sd ^(av)(m))2)/(n−1)  Eq. 4

(4) Differential Shape Descriptor

The differential shape descriptor shows the change of two consecutiveshape descriptors in a shape descriptor sequence. The differential shapedescriptor sequence DSD={dsd₁, dsd₂, dsd₃, . . . , dsd_(n−1)} can beobtained from the shape descriptor SD={sd1, sd2, sd3, . . . , sdn} basedon Equation 5.dsd _(r)=(sd _(r+1) *P _(r+1))(sd _(r) *p _(r))  Eq. 5wherein r is in the range of 0<r<n, and p_(r) denotes a weight of anr^(th) shape descriptor sd_(r), which can be obtained from a time rateof a shape represented by a shape descriptor occupying in the entireshape sequence.

The mean shape descriptor, variance shape descriptor and standarddeviation shape descriptor, i.e., (1), (2) and (3), are obtained fromthe differential shape descriptor sequence DSD={dsd₁, dsd₂, dsd₃, . . ., dsd_(n−1)} and used for abstracting motion picture shape descriptors.

The motion picture shape descriptors, suggested in the present inventioncan use the above shape descriptors alone or a combination thereof. Themotion picture shape descriptors abstracted by using a combination ofthe shape descriptors can be expressed as:CSSD={cssd ₁ , cssd ₂ , cssd _(i) , . . . , cssd _(l)}.wherein cssd₁ is one of motion picture shape descriptors suggested inthe present invention.

If a still picture shape descriptor which is irrespective of the changein size and rotation is applied, a motion picture shape descriptor alsoirrespective of the change in size and rotation is obtained.

This method of abstracting a statistically processed motion pictureshape descriptor can be used with respect to other still picturedescriptors, such as still picture texture descriptor, other than theshape descriptors used in the embodiment of the present invention.Therefore, the technology of the present invention has an advantage thatit can be generalized.

Other objects and aspects of the invention will become apparent from thefollowing description of the embodiments with reference to theaccompanying drawings, which is set forth hereinafter.

FIG. 1 is a block diagram showing a motion picture shape descriptorapparatus and a motion picture retrieving system in accordance with anembodiment of the present invention.

As described in FIG. 1, the motion picture retrieving system includes: afirst motion picture shape descriptor abstracting unit 130, a motionpicture retrieving device 110, a motion picture database (DB) 120 and amotion picture shape descriptor metadata DB 150. The motion pictureretrieving device 110 includes a second motion picture shape descriptorabstracting unit 130 a, a motion picture shape descriptor similaritycomputing unit 111 and a distance-based classification unit 112.

Hereinafter, the operation of each element will be described.

When segmented motion picture 120 is inputted by a user, the motionpicture shape descriptors for the segmented motion picture 120 areabstracted. The abstracted motion picture shape descriptors are inputtedto the motion picture shape descriptor similarity computing unit of themotion picture retrieving device 110.

The motion picture stored in the motion picture DB 120 for storingmotion pictures is inputted to the second motion picture shapedescriptor abstracting unit 130 a in the motion picture retrievingdevice 110. Then, the information outputted from the second motionpicture shape descriptor abstracting unit 130 a is stored in the motionpicture shape descriptor metadata DB 150 in the form of metadata. Themotion picture shape descriptor similarity computing unit 111 calculatesthe difference (i.e., similarity) between the motion picture shapedescriptors outputted from the first motion picture shape descriptorabstracting unit 130 and the motion picture shape descriptors in themotion picture shape descriptor metadata DB 150. To calculate thesimilarity (i.e., distance), a method using Euclidian distance whichmeasures the distance between two vectors or a method using the sum ofabsolute difference is used. The distance-based classification unit 112sorts out the calculated distance information in the order of distancesfrom close to far, abstracts corresponding metadata information from themotion picture shape descriptor metadata DB 150, and outputs theabstracted similar motion picture information 140 to the user.

FIG. 2 is a block diagram illustrating the motion picture shapedescriptor abstracter of FIG. 1 in accordance with the embodiment of thepresent invention. As illustrated in the drawing, the motion pictureshape descriptor abstracting unit 230 of the present invention includes:a motion picture segmentation unit 210, a motion picture shapedescriptor abstracting unit 230 and a motion picture metadata DB 250.The motion picture shape descriptor abstracting unit 230 includes ashape abstracter 231, a shape vector descriptor abstracter 233 and astatistical shape vector descriptor abstracter 235.

Hereinafter, the operation of each element will be described. First, amotion picture 200 is inputted to the motion picture segmentation unit210 and segmented temporally. The temporally segmented motion picture200 is inputted to the shape abstracter 231, which then outputs shapeinformation motion picture 232, corresponding to one object. The shapeinformation of each still picture of the shape information motionpicture 232 is inputted to the shape vector descriptor abstracter 233,which outputs a shape vector descriptor sequence 234.

The shape vector descriptor sequence 234 outputted from the shape vectordescriptor abstracter 233 is inputted to the statistical shape vectordescriptor abstracter 235. The statistical shape vector descriptorabstracter 235 outputs a motion picture shape descriptor 240 eventually,by using each or a combination of the Equations 1 through 5, each ofwhich corresponds to the above enumerated (1) mean shape descriptor, (2)variance shape descriptor, (3) standard deviation shape descriptor and(4) differential shape descriptor. The motion picture shape descriptor240 is stored in the motion picture metadata DB 250 for storing motionpicture metadata.

FIG. 3 is a table showing the metadata stored in a motion picturemetadata database for storing motion picture shape descriptors inaccordance with the embodiment of the present invention. The metadataare classified based on motion picture shape vector descriptor, motionpicture title, location of file, and the location of starting time inthe original motion picture.

FIG. 4 is a flow chart describing a method for abstracting a motionpicture shape descriptor in accordance with the embodiment of thepresent invention. As shown in the drawing, to abstract a motion pictureshape descriptor, at step S403, an input motion picture 400 is segmentedtemporally, and at step S405, a shape information of motion picturecorresponding to one object is abstracted from the temporally segmentedmotion picture.

Subsequently, at step S407, a shape vector descriptor sequence isabstracted from the abstracted shape information of motion picture. Atstep S409, a motion picture shape descriptor, which is a statisticalshape descriptor, is abstracted from the shape vector descriptorsequence. Then, at step S411, the abstracted motion picture shapedescriptor is stored in the motion picture metadata DB for storingmotion picture metadata.

As described above, the technology of the present invention can storethe changing shape information of a motion picture object effectively byusing a motion picture shape descriptor, and using the stored motionpicture information for retrieving motion picture and, further, forvideo indexing.

While the present invention has been described with respect to certainpreferred embodiments, it will be apparent to those skilled in the artthat various changes and modifications may be made without departingfrom the scope of the invention as defined in the following claims.

1. A system for retrieving motion picture, comprising: a motion picturesegmentation means for segmenting motion picture temporally; a motionpicture shape descriptor abstracting means for abstracting a motionpicture shape descriptor from the segmented motion picture; and a motionpicture metadata storing means for storing the motion picture shapedescriptor as metadata.
 2. The system as recited in claim 1, wherein themotion picture shape descriptor abstracting means includes: a shapeabstracting means for abstracting shape information corresponding to oneobject from the segmented motion picture; a shape vector descriptorabstracting means for abstracting shape vector descriptor sequence fromthe shape information; and a statistical shape vector descriptorabstracting means for abstracting a motion picture shape descriptor fromthe shape vector descriptor sequence.
 3. The system as recited in claim2, wherein the statistical shape vector descriptor abstracting meansabstracts motion picture shape descriptor by using one or combination ofa mean shape descriptor, a variance shape descriptor, a standarddeviation shape descriptor and a differential shape descriptor.
 4. Thesystem as recited in claim 3, wherein the mean shape descriptor isobtained based on an Equation as:sd ^(av)(m)=(Σ_(i=1 to n) sd _(i)(m))/n wherein sd_(i){sd_(i)(1),sd_(i)(2), sd_(i)(3), . . . , sd_(i)(m)}.
 5. The system as recited inclaim 3, wherein the variance shape descriptor is obtained based on anEquation as:sd ^(var)(m)=(Σ_(i=1 to n) (sd _(i)(m)−sd ^(av)(m))²)/n/(n−1) whereinsd^(av)(m)=(Σ_(i=1 to n) sd_(i)(m))/n, and sd_(i)={sd_(i)(1), sd_(i)(2),sd_(i)(3), . . . , sd_(i)(m)}.
 6. The system as recited in claim 3,wherein the standard deviation shape descriptor is obtained based on anEquation as:sd ^(std)(m)=sqrt(Σ_(i=1 to n) (sd _(i)(m)−sd ^(av)(m))2)/n/(n−1)wherein sd^(av)(m)=(Σ_(i=1 to n) sd_(i)(m))/n, and sd_(i)(m)={sd_(i)(1),sd_(i)(2), sd_(i)(3), . . . , sd_(i)(m)}.
 7. The system as recited inclaim 3, wherein the differential shape descriptor is obtained based onan Equation as:dsd _(r)=(sd _(r+1) *p _(r+1))(sd _(r) *p _(r)) wherein sd_(r) denotes ashape descriptor abstracted from the m^(th) shape information s_(r); ris in the range of 0<r<n; and p_(r) denotes a weight of the r^(th) shapedescriptor sd_(r).
 8. A system for retrieving motion picture,comprising: a first motion picture shape descriptor abstracting meansfor abstracting a first motion picture shape descriptors for motionpicture; a motion picture storing means for storing the motion picture;a motion picture shape descriptor metadata storing means for storing thefirst motion picture shape descriptor; and a motion picture retrievingmeans for calculating the similarity between the first motion pictureshape descriptor abstracted from the motion picture shape descriptorabstracting means and a second motion picture shape descriptor outputtedfrom the motion picture shape descriptor metadata storing means,arranging the motion picture shape descriptor in the order of similarityfrom small to large, and outputting similar motion pictures.
 9. Thesystem as recited in claim 8, wherein the motion picture retrievingmeans includes: a second motion picture shape descriptor abstractingmeans for abstracting motion picture shape descriptor from the. motionpicture outputted from the motion picture storing means and storing theabstracted motion picture shape descriptor in the motion picture shapedescriptor metadata storing means; a motion picture shape descriptorsimilarity computing means for calculating the similarity between afirst motion picture shape descriptor outputted from the first motionpicture shape descriptor abstracting means and the second motion pictureshape descriptor outputted from the motion picture shape descriptormetadata storing means; and a distance-based classification means forclassifying the similarity outputted from the motion picture shapedescriptor similarity computing means and outputting the similar motionpictures.
 10. The system as recited in claim 9, wherein thedistance-based classification means classifies the similarity in theorder of distance from close to far.
 11. The system as recited in claim9, wherein the motion picture shape descriptor similarity computingmeans computes the similarity based on an Euclidian distance between twoinput information vectors, or a sum of absolute differences.
 12. Amethod for abstracting a motion picture shape descriptor havingstatistical characteristics of still picture shape descriptors to beapplied to a motion picture shape descriptor abstracting apparatus, themethod comprising the steps of: a) segmenting a motion picturetemporally and abstracting shape information corresponding to one objectfrom the temporally segmented motion picture; b) abstracting a motionpicture shape descriptor, which is a statistical shape vectordescriptor, from the shape information; and c) storing the motionpicture shape descriptor in a motion picture metadata storing means. 13.The method as recited in claim 12, further comprising the steps of: d)abstracting a shape vector descriptor sequence from the abstracted shapeinformation of motion picture in order to abstract the motion pictureshape descriptor; and e) abstracting a motion picture shape descriptor,which is a statistical shape vector descriptor, from the shape vectordescriptor sequence.
 14. The method as recited in claim 13, wherein themotion picture shape descriptor, which is a statistical shape vectordescriptor of the step e), can be obtained based on an Equation as:sd ^(av)(m)=(Σ_(i=1 to n) sd _(i)(m))/n, wherein sd_(i)={sd_(i)(1),sd_(i)(2), sd_(i)(3), . . . , sd_(i)(m)}.
 15. The method as recited inclaim 13, wherein the motion picture shape descriptor can be obtainedbased on an Equation as:sd ^(var)(m)=(Σ_(i=1 to n) (sd _(i)(m)−sd ^(av)(m))²)/n/(n−1) whereinsd^(av)(m)=(Σ_(i=1 to n) sd_(i)(m))/n, and sd_(i)={sd_(i)(1), sd_(i)(2),sd_(i)(3), . . . , sd_(i)(m)}.
 16. The method as recited in claim 13,wherein the motion picture shape descriptor can be obtained based on anEquation as:sd ^(std)(m)=sqrt(Σ_(i=1 to n) (sd _(i)(m)−sd ^(av)(m))2)/(n−1) whereinsd^(av)(m)=(Σ_(i=1 to n) sd_(i)(m))/n, and sd_(i)(m)={sd_(i)(1),sd_(i)(2), sd_(i)(3), . . . , sd_(i)(m)}.
 17. The method as recited inclaim 13, wherein the motion picture shape descriptor can be obtainedbased on an Equation as:dsd _(r)=(sd _(r+1) *p _(r+1))(sd _(r) * p _(r)) wherein sdr denotes ashape descriptor abstracted from the m^(th) shape information s_(r); ris in the range of 0<r<n; and p_(r) denotes a weight of the r^(th) shapedescriptor sd_(r).
 18. A computer-based recording medium for recording aprogram for executing a method for abstracting motion picture shapedescriptors, the method comprising the steps of: a) segmenting a motionpicture temporally and abstracting shape information corresponding toone object from the temporally segmented motion picture; b) abstractinga motion picture shape descriptor, which is a statistical shape vectordescriptor, from the shape information; and c) storing the motionpicture shape descriptor in a motion picture metadata storing means,wherein the program is implemented in a motion picture shape descriptorabstracting apparatus provided with a processor.